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DETAILED ACTION 



1 . This communication is responsive to an amendment filed on 9/27/04. 

2. Claims 2, 3, 5-10, 12-16, 31, 33 and 34 are pending in this application with 
claims 1,11 and 1 7 being independent claims. Claims 1,4, 11 and 32 have been 
cancelled; claims 2, 3, 5-7, 9, 10, 12 and 13 have been amended; and claims 33 and 34 
have been added. This action is made Final. 

3. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

Claim Objections 

4. Claims 3, 1 3 and 34 are objected to because of the following informalities: 
Claim 3 recites the limitation "the input of data" in line 2. There is insufficient 

antecedent basis for this limitation in the claim. 

Claim 13 recites the limitation "the same naked hand or finger acquired from the 
camera" in line 4, which will be interpreted to mean the naked hand or finger acquired 
from the camera. There is insufficient antecedent basis for this limitation in the claim. 

Claim 15 recites the limitations "the position, viewing angles sensitivity, 
and... camera" in lines 2-3 and "the analysis" in line 3. There are insufficient antecedent 
bases for these limitations in the claims. 
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Claim 34 recites the limitations "the human user" in lines 6 and 1 1 . There are 
insufficient antecedent bases for these limitations in the claims. Appropriate 
corrections are required. 

Claim Rejections - 35 USC § 112, first paragraph 

5. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

6. Claims 33 and 34 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. New subject matter "excluding the display" on 
line 6 of claims 33 and claim 34 was not properly described in the application as filed. 
Therefore, the new subject matter will not be treated. 

Claim 7 is rejected under 35 U.S.C. 1 1 2, first paragraph, as failing to comply with 
the enablement requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to enable one skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and/or use the invention. 
The claimed subject matter for which the specification is not enabling is: "a vector on the 
plane of the display" in line 2 of page 2. 
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Claim Rejections - 35 USC § 112, second paragraph 

7. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

8. Claims 2, 7, 8, 9, 12-16 and 34 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

Claim 2 recites the limitation "the positioning of a pointing icon on the display" in 
lines 2-3. The examiner will interpret "the positioning of a pointing icon on the display" 
to mean a position of a pointing icon on the display. There is insufficient antecedent 
basis for this limitation in the claim. 

Claims 1 5 and 1 6 depend on cancelled claim 1 1 . The examiner will treat claims 
15 and 16 as depending upon claim 34. 

Claim 34 recites the limitations "based on the image coordinates of the detected 
positions" of line 1 5. The examiner will interpret "based on the image coordinates of the 
detected positions" to mean based on the detected positions. There are insufficient 
antecedent bases for these limitations in the claims. Furthermore, it is unclear what is 
meant by the limitations "detecting the positions that the user has pointed to in the 
images captured by the camera" of claim 34(e) and "based on the image coordinates of 
the detected positions., setting the boundaries of the virtual display space" of claim 34(f). 
The examiner will interpret claim 34(e) to mean "detecting the positions that the user 
has pointed to" and claim 34(f) to mean "based on the detected positions, setting the 
boundaries of the virtual display space". 
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The examiner has given applicant examples of the errors that occur in claims 2, 
3, 7, 13, 15, 16 and 34. These errors should be used as a template when reviewing 
claims 2, 5-10, 12, 14, 31 and 33. 

Claim Rejections - 35 USC §103 

9. Claims 2, 3, 5, 6, 8, 10, 31 and 33 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Rekimoto et al. ("Rekimoto") in view of Lyons, and further in view of 
Rekimoto's admitted prior art. 

As per claim 33, Rekimoto teaches a display system for use with a device having 
a display and for interacting with the device through the display, the system comprising: 

a camera, capable of sensing the spectrum of light invisible to the naked human 
eye, the camera positioned to capture an image including at least one naked finger or 
naked hand of a human user (figs. 10 and 14(A-B); col. 21, line 65 through col. 22, line 
3; invisible such as infrared light)] 

a means for identifying at least one naked finger or naked hand of the human 
user in the image captured by the camera as a pointer and a means for establishing a 
mapping between the positions that the human user's naked finger or hand is pointing 
to and the corresponding region of the display of the device (Rekimoto: figs. 4 and 6; 
col. 1, line 57 through col. 2, line 2; col. 7, lines 53 through col. 8, line 3). 

Rekimoto's main invention does not teach a camera, capable of sensing the 
spectrum of light visible to the naked human eye; however, in Rekimoto's disclosure of 
the prior art, a camera is disclosed, capable of sensing the spectrum of light visible to 
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the naked human eye (col. 1, lines 18-21; e.g. light emitting pen). Therefore, it would 
have been obvious to an artisan at the time of the invention to include Rekimoto's 
admitted prior art teaching of a camera capable of sensing the spectrum of light visible 
to the naked human eye to Rekimoto's teaching of a camera capable of sensing the 
spectrum of light /^visible to the naked human eye in order to provide users with a 
remote input that does not require user proximity to the screen. 

Rekimoto still does not explicitly disclose a means for defining a virtual display 
space around the finger or hand of the human user in the image captured by the 
camera. Lyons teaches a means for defining a virtual display space around the finger or 
hand of the human user in the image captured by the camera (col. 6, lines 42-55). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Lyons teaching of defining a virtual display space around the finger or hand of 
the human user in the image captured by the camera to Rekimoto's teaching of the 
finger or hand of the human user being captured by the camera given that virtual reality 
applications permit exciting interaction between a user and a computer and integrating 
the immersive with the non-immersive approach provides the advantage of giving the 
user the impression of being "in" the virtual reality environment using normal body 
movements. 

As per claim 2, the modified Rekimoto teaches a display system for use with a 
device having a display and for interacting with the device through the display 
comprising controlling/commanding a position of a pointing icon on the display 
(Rekimoto: fig. 19; col. 18, lines 41 -51), 
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As per claim 3, the modified Rekimoto teaches a display system for use with a 
device having a display and for interacting with the device through the display 
comprising means for controlling/commanding an input of data into the device using the 
display (Rekimoto: fig. 19; col. 18, lines 41-51). 

As per claim 5, the modified Rekimoto teaches a display system for use with a 
device having a display and for interacting with the device through the display 
comprising means for selecting different regions of the display by changing a position, 
attitude, or presentation (Rekimoto: figs. 5 and 7-10). 

As per claim 6, the modified Rekimoto teaches a display system for use with a 
device having a display and for interacting with the device through the display 
comprising means for selecting, highlighting, or defining a particular point or region on 
the display (Rekimoto: col. 9, lines 21-29). 

As per claim 8, the modified Rekimoto teaches a display system for use with a 
device having a display and for interacting with the device through the display wherein 
the pointing icon on the display can be registered by the camera that has in its field of 
view the naked hand or finger of the human user (Rekimoto: fig. 1,2,11-13 and 1 5-22). 

As per claim 10, the modified Rekimoto teaches a display system for use with a 
device having a display and for interacting with the device through the display 
comprising: 

a) means for selecting or highlighting a specific item or icon on the display 
(Rekimoto: col. 9, lines 21-29); 
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b) means for activating a specific process, program, or menu item represented 
on the display (Rekimoto: fig. 5; col. 9, lines 21-29); and 

c) means for writing, scribing, drawing, highlighting, annotating, or otherwise 
producing marks on the display (Rekimoto: fig. 5; col. 9, lines 21-29). 

As per claim 31 , the modified Rekimoto teaches a display system for use with. a 
device having a display and for interacting with the device through the display 
comprising another camera and wherein the pointing icon on the display can be 
registered by the other camera (Rekimoto: fig. 12, elements 4A and 4B). 
10. Claims 12-16 and 34 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Lyons in view of Rekimoto et al. ("Rekimoto"), and further in view of Rekimoto's 
admitted prior art. 

As per claim 34, Lyons teaches a method for defining a virtual display space for a 
human user of a device to interact with the device through its display, comprising the 
steps of: 

positioning a camera to capture an image including at least one naked finger or 
naked hand of the human user (fig. 7; described is a camera-based gesture input 
environment wherein motion of the limbs/extremities/hands/fingers or body is indicative 
of a desired operation)] 

identifying the at least one naked finger or naked hand of the human user in the 
image captured by the camera as a pointer (fig. 7; a representation of the user's 
gestures, i.e. the limbs/extremities/hands/fingers or body, appears on a display screen, 
allowing for selection of commands)] 
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asking the human user to position the limbs or body to point to specific points on 
the display (col. 12, lines 17-20; col. 13, lines 10-23); 

capturing images of the human user as the user points to the specific points (col. 
13, lines 10-23); 

detecting the positions that the user has pointed to from the images captured by 
the camera (figs. 5 and 7; col. 15, lines 8-10); 

based on the detected positions, setting a boundary of the virtual display space 
and designating within the boundary, a continuous finite space to be the virtual display 
space (figs. 5 and 7; col. 3, lines 46-56; col. 4, line 44 through col. 4, line 16; col. 15, 
lines 8-10). 

Lyons does not explicitly disclose positioning a naked finger or naked hand to 
point to specific points on the display. Rekimoto teaches a method for the human user 
of a device to interact with the device through its display comprising positioning a naked 
finger or naked hand to point to specific points on the display (figs. 10 and 14(A-B)). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Rekimoto's teaching of positioning a naked finger or naked hand to point to 
specific points on the display to Lyons teaching of positioning the limbs or body to point 
to specific points on the display in order to provide the advantage of giving the user the 
impression of being "in" the virtual reality environment using normal body movements. 

However, neither Rekimoto's main invention nor Lyon teaches a camera, capable 
of sensing the spectrum of light visible to the naked human eye; however, in Rekimoto's 
disclosure of the prior art, a camera is disclosed, capable of sensing the spectrum of 
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light visible to the naked human eye (Rekimoto: col. 1, lines 18-21 ; e.g. light emitting 
pen). Therefore, it would have been obvious to an artisan at the time of the invention to 
include Rekimoto's admitted prior art teaching of a camera capable of sensing the 
spectrum of light visible to the naked human eye to Rekimoto's teaching of a camera 
capable of sensing the spectrum of light /^visible to the naked human eye in order to 
provide users with a remote input that does not require user proximity to the screen. 

As per claim 12, the modified Lyons teaches a method for defining a virtual 
display space for a human user of a device to interact with the device through its display 
wherein at least one characteristic that distinguishes the pointing object/naked hand or 
finger from other objects in the image registered by the camera is known a priori 
(Rekimoto: fig. 19; col, 18, line 52 through col. 16, line 44; Lyons: col. 1, lines 46-56; col. 
3, lines 46-50; col. 13, lines 27-42). 

As per claims 13 and 14, the modified Lyons teaches a method for defining a 
virtual display space for a human user of a device to interact with the device through its 
display wherein at least one characteristic that distinguishes the pointing object/naked 
hand or finger from other objects in the image from the camera is determined based on 
analysis of at least one image of the pointing object/naked hand or finger acquired from 
the camera (Lyons: figs. 5 and 7) and wherein at least one characteristic that 
distinguishes the pointing object/naked human hand or finger from other objects, whose 
rendition are present in the image from the camera is obtained by acquiring at least two 
images from the camera, one with the pointing object in view of the camera and one 
without, and comparing the two sets with one another (Rekimoto: figs. 1 ,7-9 and 14(A- 
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B); col. 7, lines 5-11; col. 7, line 49 through col. 8, line 57; col. 9, lines 60-65; col. 10, 
lines 45-60; col. 15, line 60 through col. 16, line 36). 

As per claim 15, the modified Lyons teaches a method for defining a virtual 
display space for a human user of a device to interact with the device through its display 
wherein adjustments or modifications are made to position, viewing angles, sensitivity, 
and other settings of the camera pursuant an analysis of the data or image retrieved 
from the camera (Lyons: col. 2, lines 32-35; col. 12, lines 35-44; Rekimoto: col. 2, lines 
24-42). 

As per claim 16, the modified Lyons teaches a method for defining a virtual 
display space for a human user of a device to interact with the device through its display 
wherein at least part of the procedures for the method is carried out using at least in 
part a computing mechanism available on one or more of the following: the display, or 
the camera, or the pointing device, or the device producing the signal shown on the 
display, or the device producing the pointing icon on the display (Rekimoto: figs. 1 ,7-9 
and 1 4(A-B); col. 7, lines 5-1 1 ; col. 7, line 49 through col. 8, line 57; col. 9, lines 60-65; 
col. 10, lines 45-60; col. 15, line 60 through col. 16, line 36). 

11. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Rekimoto 
et al. ("Rekimoto") in view of Lyons as applied to claim 33, and further in view of 
Edwards et al. ("Edwards"). 

As per claim 7, the modified Rekimoto teaches a display system for use with a 
device having a display and for interacting with the device through the display 
comprising a naked hand or finger for entering information into the system and can be 
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used in conjunction with various software applications (Rekimoto: figs. 1,7-9 and 14(A- 
B); col. 7, lines 5-1 1 ; col. 7, line 49 through col. 8, line 57; col. 9, lines 60-65; col. 1 0, 
lines 45-60; col. 15, line 60 through col. 16, line 36). The modified Rekimoto does not 
explicitly disclose the pointing object/naked hand or finger to be used to define a vector 
on the plane of the display that indicates a direction and magnitude relative to or with 
respect to an item on the display or a region of the display. Edwards teaches a system 
for interacting with displays wherein the pointing object to be used to define a vector on 
the plane of the display that indicates a direction and magnitude relative to or with 
respect to an item on the display or a region of the display (col. 8, lines 12-21 ). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Edwards 5 teaching of a computer aided design tool, wherein the pointing object 
to be used to define a vector on the plane of the display that indicates a direction and 
magnitude relative to or with respect to an item on the display or a region of the display, 
to the modified Rekimoto's teaching of a system for interacting with displays that 
includes drawing capabilities to provide users with an environment capable of 
incorporating multiple applications and capabilities to enhance a user's individual task. 
12. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Rekimoto 
et al. ("Rekimoto") in view of Lyons as applied to claim 2, and further in view of 
Applicant's Admitted prior art. 

As per claim 9, although the modified Rekimoto "teaches a display system for use 
with a device having a display and for interacting with the device through the display 
comprising means for sensing the naked finger/pointing object's position relative to the 
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position of a pointer icon on the display (Lyons: fig. 5; col. 7, lines 39-55), the modified 
Rekimoto does not explicitly disclose the system to include a means for correcting the 
offsets between the position of the naked finger/pointing object, or reflection, or effect 
thereof on the display as observed by the user or by the camera, and the position of the 
pointer icon on the display. However, Applicant's admitted prior art teaches a system for 
interacting with displays and all devices that use such displays to include a means for 
correcting the offsets between the position of the pointing object, or reflection, or effect 
thereof on the display as observed by the user or by the sensor or the camera, and the 
position of the pointer icon on the display (page 21, lines 6-9). Therefore, it would have 
been obvious to an artisan at the time of the invention to include Applicant admitted 
prior art's teaching of a system for correcting the offsets between the position of the 
pointing object and the position of the pointer icon on the display to the modified 
Rekimoto's system for sensing the naked finger/pointing object's position relative to the 
position of a pointer icon on the display in order to reduce the margin of error 
concerning the position of the pointing object and the position of the pointer icon on the 
display. 

Response to Arguments 

13. Applicant's arguments with respect to claims 33 and 34 have been considered 
but are moot in view of the new ground(s) of rejection. 
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Conclusion 

14. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Oh (US 5,616,078) teaches a motion-controlled video entertainment system. 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 
706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 
1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Inquires 

15. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Examiner Le Nguyen whose telephone number is (571) 
272-4068. The examiner can normally be reached on Monday - Friday from 7:00 am to 
3:30 pm (EST). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid, can be reached on (703) 308-0640. 

The fax numbers for the organization where this application or proceeding is 
assigned are as follows: 

(703) 872-9306 [Official Communication] 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 305- 
3900. 



LVN 

Patent Examiner 
July 21, 2005 



KRISTIN- ^'CAID 
SUPERVISOR PAT6MT 6XAMINER 
TECHNOL0&V OX&L 2100 



